Field
The invention relates to image segmentation devices and related methods of automatically segmenting organ chambers using deep learning methods to compute clinical indices from imaging data.
Description of the Related Art
Cardiac magnetic resonance imaging (MRI) is now routinely being used for the evaluation of the function and structure of the cardiovascular system (Yuan, C. et al. 2002 J Magn Reson Imaging 15(1): 62-67; Lima, J. C. and Desai, M. Y. 2004. J Am Coll Cardiol 44(1): 1164-1171; Frangi, A. F. et al. 2001 IEEE Trans Med Imag 20(1): 2-5; Petitjean, C. and Dacher, J.-N. 2011 Med Image Anal 15(2): 169-184; Tavakoli, V. and Amini, A. A. 2013 Comput Vis Image Underst 117(9); 966-989; Heimann, T. and Meinzer, H.-P. 2009 Med Image Anal 13(4): 543-563; Suinesiaputra, A. et al. 2014 Med Image Anal 18(1): 50-62). Segmentation of the left ventricle (LV) from cardiac MRI datasets is an essential step for calculation of clinical indices such as ventricular volume, ejection fraction, left ventricular mass and wall thickness as well as analyses of the wall motion abnormalities.
Manual delineation by experts is currently the standard clinical practice for performing the LV segmentation. However, manual segmentation is tedious, time consuming and prone to intra- and inter-observer variability (Frangi, A. F. et al. 2001 IEEE Trans Med Imag 20(1): 2-5; Petitjean, C. and Dacher, J.-N. 2011 Med Image Anal 15(2): 169-184; Tavakoli, V. and Amini, A. A. 2013 Comput Vis Image Underst 117(9); 966-989; Heimann, T. and Meinzer, H.-P. 2009 Med Image Anal 13(4): 543-563; Suinesiaputra, A. et al. 2014 Med Image Anal 18(1): 50-62). To address this, it is necessary to reproducibly automate this task to accelerate and facilitate the process of diagnosis and follow-up. To date, several methods have been proposed for the automatic segmentation of the LV. A review of these methods can be found in Frangi, A. F. et al. 2001 IEEE Trans Med Imag 20(1): 2-5; Petitjean, C. and Dacher, J.-N. 2011 Med Image Anal 15(2): 169-184; Tavakoli, V. and Amini, A. A. 2013 Comput Vis Image Underst 117(9); 966-989; Heimann, T. and Meinzer, H.-P. 2009 Med Image Anal 13(4): 543-563; Suinesiaputra, A. et al. 2014 Med Image Anal 18(1): 50-62.
To summarize, there are several challenges in the automated LV segmentation from cardiac MRI datasets: heterogeneities in the brightness of LV cavity due to blood flow; presence of papillary muscles with signal intensities similar to that of the myocardium; complexity in segmenting the apical and basal slice images; partial volume effects in apical slices due to the limited resolution of cardiac MRI; inherent noise associated with cine cardiac MRI; dynamic motion of the heart and inhomogeneity of intensity; considerable variability in shape and intensity of the heart chambers across patients, notably in pathological cases, etc. (Tavakoli, V. and Amini, A. A. 2013 Comput Vis Image Underst 117(9); 966-989; Petitjean, C. and Dacher, J.-N. 2011 Med Image Anal 15(2): 169-184; Queiros, S. et al. 2014 Med Image Anal 18(7): 1115-1131). Due to these technical barriers, the task of automatic segmentation of the heart chambers from cardiac MRI is still a challenging problem (Petitjean, C. and Dacher, J.-N. 2011 Med Image Anal 15(2): 169-184; Tavakoli, V. and Amini, A. A. 2013 Comput Vis Image Underst 117(9); 966-989; Suinesiaputra, A. et al. 2014 Med Image Anal 18(1): 50-62).
Current approaches for automatic segmentation of the heart chambers can be generally classified as: pixel classification (Kedenburg, G. et al. 2006 in Automatic Cardiac MRI Myocardium Segmentation Using Graphcut. International Society for Optics and Photonics, p. 61440A; Cocosco, C. A. et al. 2008 J Magn Reson Imaging 28(2): 366-374), image-based methods (Jolly, M., 2009 MIDAS J.—Cardiac MR Left Ventricle Segm Chall 4; Liu, H. 2012 Academic Radiology 19(6): 723-731), deformable methods (Billet, F. et al. 2009 Funct Imaging Model Heart 376-385; Ben Ayed, I. et al. 2009 IEEE Trans Med Imaging 28(12): 1902-1913; Chang, H.-H. et al. 2010 IEEE Trans Vis Comp Gr 16(5): 854-869; Pluempitiwiriyawej, C. et al. 2005 IEEE Trans Med Img 24(5): 593-603), active appearance and shape models (AAM/ASM) (Zhang, H. et al. 2010 IEEE Trans Med Img 29(2): 350-364; Assen, H. C. et al. 2006 Med Image Anal 10(2): 286-303) and atlas models (Zhuang, X. et al. 2008 Robust Registration Between Cardiac MRI Images and Atlas for Segmentation Propagation. International Society for Optics and Photonics, p. 691408; Lorenzo-Valdés, M. et al. 2004 Med Image Anal 8(3): 255-265). Pixel classification, image-based and deformable methods suffer from a low robustness and accuracy and require extensive user interaction (Petitjean, C. and Dacher, J.-N. 2011 Med Image Anal 15(2): 169-184). Alternatively, model-based methods such as AAM/ASM and atlas models can overcome the problems related to the previous methods and reduce user interaction at the expense of a large training set to build a general model. However, it is very difficult to build a model that is general enough to cover all possible shapes and dynamics of the heart chambers (Petitjean, C. and Dacher, J.-N. 2011 Med Image Anal 15(2): 169-184; Jolly, M. et al. 2009 in Combining Registration and Minimum Surfaces for the Segmentation of the Left Ventricle in Cardiac Cine MR Images, vol. 5762. Springer Berlin Heidelberg, pp. 910-918). Small datasets lead to a large bias in the segmentation, which makes these methods inefficient when the heart shape is outside the learning set (e.g., congenital heart defects, post-surgical remodeling, etc.).
Furthermore, current learning-based approaches for LV segmentation have certain limitations. For instance, methods using random forests (Margeta, J. et al. 2012. Stat Atlases Comput Models Heart Imaging Model. Chall 109-119; Lempitsky, V. et al. 2009 Funct Imaging Model Heart 447-456; Geremia, E. et al. 2011 NeuroImage 57(2): 378-390) rely on image intensity and define the segmentation problem as a classification task. These methods employ multiple stages of intensity standardization, estimation and normalization, which are computationally expensive and affect the success of further steps. As such, their performance is rather mediocre at basal and apical slices and overall inferior to the state-of-the-art. Also, methods that use Markov random fields (MRFs) (Cordero-Grande, L., et al. 2011 Med Image Anal 15(3): 283-301; Huang, R. et al. 2004 Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2004, 2, pp. 11-739), conditional random fields (CRFs) (Cobzas, D. and Schmidt, M. 2009 In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 328-335; Dreijer, J. et al. 2013 BMC Med Imaging 13 (1). doi: 10.1186/1471-2342-13-24) and restricted Boltzman machines (RBMs) (Ng, A. “The deep learning tutorial. Ngo, T. A., Carneiro, G., 2014. Fully automated non-rigid segmentation with distance regularized level set evolution initialized and constrained by deep-structured inference.” The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 3118-3125) have been considered. MRF and RBM are generative models that try to learn the probability of input data. Computing the image probability and parameter estimation in generative models is usually difficult and can lead to reduced performance if oversimplified. Besides, they use the Gibbs sampling algorithm for training, which can be slow, become stuck for correlated inputs, and produce different results each time it is run due to its randomized nature. Alternatively, CRF methods try to model the conditional probability of latent variables, instead of the input data. However, they are still computationally difficult, due to complexity of parameter estimation, and their convergence is not guaranteed (Dreijer, J. et al. 2013 BMC Med Imaging 13 (1). doi: 10.1186/1471-2342-13-24).
Compared to left ventricle (LV), the study of the right ventricle (RV) is a relatively young field. Although many recent studies have confirmed the prognostic value of RV function in cardiovascular disease, for several years its significance has been underestimated (F. Haddad et al. 2008 Circulation 117(13): 1717-1731; J. Sanz et al. 2012 Cardiology Clinics, 30(2): 189-203). Understanding the role of RV in the pathophysiology of heart failure traditionally has dawdled behind that of the LV. The RV used to be considered a relatively passive chamber relating the systemic and pulmonary circulation until more recent studies revealed its importance in sustaining the hemodynamic stability and cardiac performance (S. R. Mehta et al. 2001 Journal of the American College of Cardiology 37(1):37-43; L. G. Kevin and M. Barnard 2007 Continuing Education in Anaesthesia, Critical Care & Pain 7(3):89-94; A. Vitarelli and C. Terzano 2010 Heart Failure Reviews 15(1): 39-61).
Cardiac magnetic resonance imaging (MRI) is the preferred method for clinical assessment of the right ventricle (C. Petitjean et al. 2015 Medical Image Analysis 19(1): 187-202; C. Yuan, et al. 2002 Journal of Magnetic Resonance Imaging 15(1):62-67; J. C. Lima and M. Y. Desai 2004 Journal of the American College of Cardiology 44(1):1164-1171; A. F. Frangi et al. 2001 IEEE Trans on Med Imaging 20(1): 2-5; V. Tavakoli and A. A. Amini. 2013 Computer Vision and Image Understanding 117(9):966-989; T. Heimann and H.-P. Meinzer et al. 2009 Medical Image Analysis 13(4):543-563; T. Geva 2014 Circ Cardiovasc Imaging 7(1): 190-197). Currently RV segmentation is manually performed by clinical experts, which is lengthy, tiresome and sensitive to intra and inter-operator variability (C. Petitjean et al. 2015 Medical Image Analysis 19(1): 187-202; J. Caudron, J. et al. 2011 European Radiology 21(10): 2111-2120, 2011; L. Bonnemains et al. 2012 Magnetic Resonance in Medicine 67(6): 1740-1746). Therefore, automating the RV segmentation process turns this tedious practice into a fast procedure. This ensures the results are more accurate and not vulnerable to operator-related variabilities, and eventually accelerates and facilitates the process of diagnosis and follow-up.
There are many challenges for RV segmentation that are mainly attributed to RV anatomy. These can be summarized as: presence of RV trabeculations with signal intensities similar to that of the myocardium, complex crescent shape of the RV, which varies from base to apex, along with inhomogeneity reflected in the apical image slices, and considerable variability in shape and intensity of the RV chamber among subjects, notably in pathological cases (C. Petitjean et al. 2015 Medical Image Analysis 19(1): 187-202).
Currently, only limited studies have focused on RV segmentation (C. Petitjean et al. 2015 Medical Image Analysis 19(1): 187-202). Atlas-based methods have been considered in some studies (M. A. Zuluaga, et al. 2013 Functional Imaging and Modeling of the Heart 7945:174-181; M. A. Zuluaga, et al. 2013 Functional Imaging and Modeling of the Heart 7945:174-181; W. Bal, et al. 2013 IEEE Transactions on Medical Imaging 32(7): 1302-1315), which require large training datasets and long computational times while their final segmentation may not preserve the mostly regular boundaries of the ground-truth masks (Y. Ou et al. 2012 “Multi-atlas segmentation of the cardiac mr rightventricle” Proceedings of 3D Cardiovascular Imaging: A MICCAI Segmentation Challenge. Nice, France). Nevertheless, it is challenging to build a model general enough to cover all possible RV shapes and dynamics (C. Petitjean and J.-N. Dacher 2011 Medical Image Analysis 15(2):169-184). Alternatively, graph-cut-based methods combined with distribution matching (C. Nambakhsh “Rapid automated 3d RVendocardium segmentation in mri via convex relaxation and distribution matching” Proc. MICCAI RV Segmentation Challenge, 2012), shape-prior (D. Grosgeorge et al. Computer Vision and Image Understanding 117(9): 1027-1035) and region-merging (O. M. O. Maier, et al. 2012 “Segmentation of RVin 4D cardiac MR volumes using region-merging graph cuts” in Computing in Cardiology (CinC), pages 697-700) were studied for RV segmentation. Overall, these methods suffer from a low robustness and accuracy, and require extensive user interaction. Image-based methods have been considered by Ringenberg et al. (J. Ringenberg, et al. 2014 Computerized Medical Imaging and Graphics 38(3): 190-201) and Wang et al. (C. Wang, et al. 2012 “A simple and fully automatic right ventricle segmentation method for 4-dimensional cardiac mr images” Proc. of 3D Cardiovascular Imaging: a MICCAI segmentation challenge, 2012). They showed notable accuracy and processing time improvement over other methods while deformed RV shape and patient movement during the scan are the limitations of their method (J. Ringenberg, et al. 2014 Computerized Medical Imaging and Graphics 38(3): 190-201). Current learning based approaches, such as probabilistic boosting trees and random forests, depend on the quality and extent of annotated training data and are computationally expensive (X. Lu, et al. 2011 Functional Imaging and Modeling of the Heart 6666(3): 250-258; D. Mahapatra and J. M. Buhmann. Automatic cardiac RV segmentation using semantic information with graph cuts. Biomedical Imaging (ISBI), 2013 IEEE 10th International Symposium on, pages 1106-1109, April 2013; O. Moolan-Feroze et al. 2014 Segmentation of the right ventricle using diffusion maps and markov random fields. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2014, pages 682-689, Springer; D. Mahapatra et al. 2014 Journal of Digital Imaging 27(6): 794-804).